1. Field of the Invention
The present invention relates to an image processing method and an image processing apparatus. Particularly, the present invention relates to an image processing method and an image processing apparatus suitable for use in dividing a plurality of image data, correction data, etc. of a main memory to handle the data in image processing to execute image processing and correction processing by sequential reading of necessary data.
2. Description of the Related Art
In general, local (near) image processing, such as a spatial filtering process, is executed to form and output an image. The local image processing is image processing for performing some kind of calculation using pixels included in a spatial filter area including pixels to be processed (hereinafter, abbreviated as processing pixels).
For example, a spatial filtering process, such as edge enhancement and shading, is applied to digital image data 300 of FIG. 3A. U.S. Pat. No. 6,587,158 (hereinafter, Document 1), Japanese Patent Laid-Open No. 2000-312327 (hereinafter, Document 2), Japanese Patent No. 3733826 (hereinafter, Document 3), and Japanese Patent Laid-Open No. 2006-139606 (hereinafter, Document 4) provide a technique for dividing digital image data into areas to apply local image processing to each area. In the technique, as shown in FIGS. 3A to 3D, the entire data of a digital image is divided into band shapes (strip shapes), and various image processes are sequentially applied to the areas.
The divided elongated areas will be called band areas, and a storage area, in which the band areas are developed, will be called a band memory. The band memory is reserved in, for example, a main memory as a storage area. In the band processing, local image processing is executed with no space between the band areas. Therefore, parts of the band areas overlap each other at boundaries with adjacent areas. In Document 4, pixels are scanned one by one in the direction of the height of the band areas to define, by the sizes of the height of the band areas, the capacity of the delay memory holding processing pixels required in the local image processing, and memory saving of the delay memory is realized.
Meanwhile, there are many image processes that cannot be handled by considering only the local image processing, as in the technique described in Documents 1 to 4.
First, there is a correction process to correct individual differences in read elements of a sensor device in an image read by an image reading apparatus such as a scanner. In the correction process, a read image to be corrected is corrected pixel data by pixel data (pixel value) based on read minimum value data and read maximum value data.
Furthermore, there is image processing for combining a plurality of rendering images in accordance with blending data (α value). In this type of image processing, for example, two rendering images to be combined are combined with a combination ratio (blending data) set in each pixel.
Furthermore, for example, a plurality of field images continuous in the time axis direction are referenced to execute an adaptive spatial filtering process as in interlace/progressive conversion (hereinafter, abbreviated as I/P conversion) of moving images. In image processing of referencing a plurality of field images, such as in the I/P conversion, high-quality image processing can be executed if an adaptive process is executed using a larger amount of field images. Therefore, it is important to flexibly switch the image processing method in accordance with the resolution and the number of reference fields.
What is common in the above image processing is that it is not sufficient just to execute image processing by considering only one piece of image data in which pixel data is two-dimensionally arranged. For example, there are the following cases of the image data, the correction data, and the blending data necessary in the image processing.
The number of times of reading from an external memory varies depending on the sensor shape.                The number of rendering images read from an external memory varies depending on the number designated by the user.        The number of field images read from an external memory vary depending on the input resolution of broadcast wave.        
As described, there are a variety of data necessary in image processing, and the data are usually stored in an external memory. Therefore, a variety of data need to be read from the external memory in accordance with the type of image processing, and the data need to be transferred to an image processing circuit.
In the field of image processing apparatus, a CPU usually sets a read address to a direct memory access controller (hereinafter, DMAC) and activates the DMAC. The activated DMAC acquires data necessary for image processing from the external memory through a memory controller in accordance with the read address and transfers the acquired data to the image processing circuit. The image processing circuit executes the image processing based on the input data.
The data is stored in another area of the external memory, because the types of data are different such as in the correction process, or because the data is inputted from the external device at different time depending on the broadcast wave such as in the I/P conversion. To further divide the data stored in the other area into the band areas to sequentially execute the image processing, the data in the other area needs to be partially read from the external memory for an amount of areas equivalent to the band areas, and the data needs to be transferred to the image processing circuit. To realize such data reading, a multi-channel DMAC as in Japanese Patent Laid-Open No. 7-320066 (hereinafter, Document 5) and Japanese Patent Laid-Open No. 2002-366507 (hereinafter, Document 6) may be used.
However, even if the multi-channel DMAC is used, in the sequential image processing of the band areas, a CPU that controls the DMAC needs to perform the following series of controls every time an image processing completion interrupt from the image processing circuit is detected.
(1) Set top addresses and transfer sizes of various data necessary in the image processing of the next band areas to the DMAC.
(2) Set the register of the image processing circuit.
(3) Activate the DMAC.
(4) Transfer image data from the external memory to the image processing circuit by the DMAC. Therefore, the CPU needs to repeat the control of (1) to (4) for the number of times of band processing, for each type of data.
To scan one pixel in the height direction of the band areas as in Documents 2 to 4, an image processing input circuit described below needs to perform HV conversion (scan conversion from the horizontal direction to the vertical direction). For the HV conversion, the CPU activates the DMAC for a significantly large number of times for low-capacity transfer blocks described below, and the number of controls by the CPU increases.
Meanwhile, a universal CPU is designed to consume processing time of one cycle for any calculation and also consume processing time of one cycle for writing and reading of parameters, intermediate calculation data, etc. that are necessary in image processing to and from the register. Therefore, the processing capacity per operating frequency of an image processing circuit designed to be able to execute a calculation process in parallel in a pipeline configuration is significantly higher.
In recent years, it is desired to handle various requests of image processing within limited cost. More specifically, an apparatus and a method for flexibly realizing the various image processes while keeping the memory bandwidth and the circuit size constant are desired.
In the conventional technique of Documents 1 to 4, no examination is performed from a viewpoint of flexibly acquiring various data from an external memory in accordance with the quantity of a plurality of correction data for one piece of image data, a plurality of image data, or a plurality of consecutive field images. Examples of the quantity of data include the number of correction data, the number of rendering images, and the number of reference fields.
In Documents 5 and 6, how to associate the CPU, the DMAC, and the image processing unit is not examined from a viewpoint of flexible acquisition, for each band area, of the various data necessary in the various image processes from the external memory. Furthermore, no examination is performed from a viewpoint of improving the processing capacity per operating frequency (consumed power) by the CPU dedicated to image processing temporarily using calculation resources of the image processing unit.